<html>
<head><meta charset="utf-8"><title>jobserver improvements · t-compiler/wg-parallel-rustc · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/index.html">t-compiler/wg-parallel-rustc</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html">jobserver improvements</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="181931318"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181931318" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181931318">(Nov 26 2019 at 15:21)</a>:</h4>
<p>quick update on jobserver work: afaict, the new(?) rayon is buggy and releases jobserver tokens it didn't acquire</p>



<a name="181931378"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181931378" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181931378">(Nov 26 2019 at 15:22)</a>:</h4>
<p>which may have led to magical performance that we were seeing, uncertain</p>



<a name="181931387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181931387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181931387">(Nov 26 2019 at 15:22)</a>:</h4>
<p>in any case, investigating to see if I can fix now</p>



<a name="181931773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181931773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181931773">(Nov 26 2019 at 15:25)</a>:</h4>
<p>As in rustc-rayon, used on master?</p>



<a name="181934878"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181934878" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181934878">(Nov 26 2019 at 15:52)</a>:</h4>
<p>no, the one we're trialing</p>



<a name="181934951"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181934951" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181934951">(Nov 26 2019 at 15:53)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> Am I correct that we expect rustc to never release a jobserver token until after it's acquired it itself? i.e., it should not release the token that cargo acquired to spawn it</p>



<a name="181934975"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181934975" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181934975">(Nov 26 2019 at 15:53)</a>:</h4>
<p>Yes</p>



<a name="181935087"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181935087" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181935087">(Nov 26 2019 at 15:54)</a>:</h4>
<p>(in particular, rayon does this before blocking -- I'm not sure yet on what)</p>



<a name="181935798"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181935798" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181935798">(Nov 26 2019 at 16:00)</a>:</h4>
<p>rustc can and will release the implicit token given to it by cargo though, which it didn't explicitly acquire.</p>



<a name="181936808"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181936808" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181936808">(Nov 26 2019 at 16:09)</a>:</h4>
<p><span class="user-mention" data-user-id="116466">@Zoxc</span> I'm confused</p>



<a name="181936823"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181936823" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181936823">(Nov 26 2019 at 16:09)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> just said that we should <em>never</em> do that</p>



<a name="181936913"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181936913" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181936913">(Nov 26 2019 at 16:10)</a>:</h4>
<p>(and I tend to agree, otherwise we can end up with a bunch of rustcs even with <code>-j1</code>)</p>



<a name="181938150"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938150" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938150">(Nov 26 2019 at 16:23)</a>:</h4>
<p>Ideally we shouldn't do it, but it's not buggy</p>



<a name="181938519"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938519" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938519">(Nov 26 2019 at 16:27)</a>:</h4>
<p>well</p>



<a name="181938568"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938568" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938568">(Nov 26 2019 at 16:28)</a>:</h4>
<p>I feel like it is buggy?</p>



<a name="181938603"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938603" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938603">(Nov 26 2019 at 16:28)</a>:</h4>
<p>i.e., if you release that token then we're basically ending up in a situation where we can deadlock, right? since no one is able to make progress</p>



<a name="181938647"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938647" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938647">(Nov 26 2019 at 16:29)</a>:</h4>
<p>you need each process to always have at least one token, otherwise it shouldn't do anything</p>



<a name="181938800"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938800" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938800">(Nov 26 2019 at 16:30)</a>:</h4>
<p>It will immediately (or before) acquire the token back with the Rayon threadpool though.</p>



<a name="181938818"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938818" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938818">(Nov 26 2019 at 16:30)</a>:</h4>
<p>but it can lose that race</p>



<a name="181938850"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938850" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938850">(Nov 26 2019 at 16:31)</a>:</h4>
<p>at which point no one is going to make progress</p>



<a name="181938857"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938857" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938857">(Nov 26 2019 at 16:31)</a>:</h4>
<p>How so?</p>



<a name="181938889"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938889" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938889">(Nov 26 2019 at 16:31)</a>:</h4>
<p>hm, okay, I should clarify -- eventually, it'll make progress</p>



<a name="181938941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938941">(Nov 26 2019 at 16:32)</a>:</h4>
<p>but it'll stall until (potentially) N other rustcs try to finish</p>



<a name="181938965"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938965" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938965">(Nov 26 2019 at 16:32)</a>:</h4>
<p>in any case it seems pretty clear that this behavior is pretty suboptimal</p>



<a name="181938985"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938985" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938985">(Nov 26 2019 at 16:32)</a>:</h4>
<p>(since it means that <code>-j1</code> is no longer guaranteed to have at most one rustc process, etc)</p>



<a name="181938997"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181938997" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181938997">(Nov 26 2019 at 16:33)</a>:</h4>
<p>and that to me even seems like just flat "buggy"</p>



<a name="181939156"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181939156" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181939156">(Nov 26 2019 at 16:35)</a>:</h4>
<p>I guess I won't try to fix it and just hope things actually work, even if I can't really tell due to this</p>



<a name="181939809"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181939809" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181939809">(Nov 26 2019 at 16:42)</a>:</h4>
<p>rustc can also "stall" in the same way due to Rayon giving up tokens to one of the threads it's using while it's idle, or due to rustc waiting on a query which is currently computing on another thread.</p>



<a name="181940420"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181940420" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181940420">(Nov 26 2019 at 16:49)</a>:</h4>
<p>It would be nice to ensure that rustc always keeps at least one token. There was code to do this in some of the cases, but I removed it since it was buggy (<a href="https://github.com/rust-lang/rust/pull/59804/files" target="_blank" title="https://github.com/rust-lang/rust/pull/59804/files">https://github.com/rust-lang/rust/pull/59804/files</a>). To fix the transfer of the jobserver token from the main thread to the thread pool would probably require some less pretty Rayon changes (an API to enter a thread pool while holding a token to be given to the thread pool).</p>



<a name="181940627"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181940627" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181940627">(Nov 26 2019 at 16:51)</a>:</h4>
<p>well, it'd mostly be a matter of using the thread pool on the side</p>



<a name="181940648"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181940648" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181940648">(Nov 26 2019 at 16:51)</a>:</h4>
<p>e.g. you'd not have the 'main' thread be a threadpool thread</p>



<a name="181940679"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181940679" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181940679">(Nov 26 2019 at 16:51)</a>:</h4>
<p>(which I think is largely true most of the time? e.g. in non-rustc land)</p>



<a name="181940724"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181940724" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181940724">(Nov 26 2019 at 16:51)</a>:</h4>
<p>It has to be a threadpool thread though, for efficiency and for <code>WorkerLocal</code> to work.</p>



<a name="181940808"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181940808" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181940808">(Nov 26 2019 at 16:52)</a>:</h4>
<p>Using it off the threadpool hits the slow paths in Rayon.</p>



<a name="181941300"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181941300" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181941300">(Nov 26 2019 at 16:58)</a>:</h4>
<p>okay, but that seems like a solveable problem</p>



<a name="181941351"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181941351" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181941351">(Nov 26 2019 at 16:58)</a>:</h4>
<p>maybe this is a good reason to not use rayon (as we've discussed a little)</p>



<a name="181941852"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181941852" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181941852">(Nov 26 2019 at 17:03)</a>:</h4>
<p>No, any other thread pool should be designed the same way and we should be running the main rustc thread on it =P</p>
<p>Maybe an API for a rustc thread to become one of the Rayon threadpool threads would work? Or just an API for spawning and joining the threadpool at once should also probably suffice.</p>



<a name="181942737"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181942737" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181942737">(Nov 26 2019 at 17:11)</a>:</h4>
<p>Actually we could just provide a parameter to the thread pool with how many jobserver tokens it starts with and skip acquiring tokens for some threads, and then have an API to enter the thread pool without giving up any tokens.</p>



<a name="181942907"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181942907" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181942907">(Nov 26 2019 at 17:13)</a>:</h4>
<p>I don't really know what the APIs etc need to be like</p>



<a name="181942924"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181942924" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181942924">(Nov 26 2019 at 17:13)</a>:</h4>
<p>but it does seem like we need to more deeply integrate token management into rayon</p>



<a name="181943998"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181943998" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181943998">(Nov 26 2019 at 17:27)</a>:</h4>
<p>I guess we can just do nothing with jobserver tokens when entering a thread pool then. So we just need a flag which tells Rayon one of its threads don't need to release / acquire a jobserver token on startup / exit</p>



<a name="181945608"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945608" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945608">(Nov 26 2019 at 17:44)</a>:</h4>
<p>Sorry haven't caught up on the discussion here, but it is incorrect for rustc to release its implicit token</p>



<a name="181945613"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945613" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945613">(Nov 26 2019 at 17:44)</a>:</h4>
<p>and that is a bug we cannot ship with</p>



<a name="181945626"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945626" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945626">(Nov 26 2019 at 17:44)</a>:</h4>
<p>part of the major purpose of the jobserver is to limit memory consumption</p>



<a name="181945633"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945633" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945633">(Nov 26 2019 at 17:44)</a>:</h4>
<p>if you release the implicit token then it's possible to spawn unlimited rustc instances</p>



<a name="181945638"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945638" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945638">(Nov 26 2019 at 17:44)</a>:</h4>
<p>and they're all just sitting idle consuming memory</p>



<a name="181945645"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945645" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945645">(Nov 26 2019 at 17:44)</a>:</h4>
<p>for example if you run with <code>-j1</code>, only one will ever be doing work</p>



<a name="181945654"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945654" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945654">(Nov 26 2019 at 17:44)</a>:</h4>
<p>but we may spawn hundreds of <code>rustc</code> instances in parallel</p>



<a name="181945671"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181945671" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181945671">(Nov 26 2019 at 17:45)</a>:</h4>
<p>which can kill builds that rely on <code>-j</code> for limiting memory</p>



<a name="181951196"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181951196" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181951196">(Nov 26 2019 at 18:46)</a>:</h4>
<p>That doesn't seem to be an easy property to guarantee</p>



<a name="181951713"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181951713" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181951713">(Nov 26 2019 at 18:51)</a>:</h4>
<p>no, it is not, and this is one reason why the llvm backend is so complicated right now, it has to juggle around this "implicit token"</p>



<a name="181952660"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181952660" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181952660">(Nov 26 2019 at 19:00)</a>:</h4>
<p>I'm guessing it would require fibers in Rayon to avoid giving up jobserver tokens when waiting on queries, but I'm not sure if that covers all cases. Fiber also can also cause excessive memory usage if not limited by waiting, so it's not a great fix.</p>



<a name="181960696"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181960696" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181960696">(Nov 26 2019 at 20:32)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> in theory <a href="https://github.com/rust-lang/rust/commit/cf92e2ebf129f4fd8075650b21c612d196b7a2f8" target="_blank" title="https://github.com/rust-lang/rust/commit/cf92e2ebf129f4fd8075650b21c612d196b7a2f8">cf92e2ebf129f4fd8075650b21c612d196b7a2f8</a> should be a valid try commit (you want alt build as usual, and you'll need to rebuild cargo -- I'm not sure if we uploaded a copy)</p>



<a name="181960731"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181960731" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181960731">(Nov 26 2019 at 20:33)</a>:</h4>
<p>or how to get that copy if we did</p>



<a name="181960780"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181960780" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181960780">(Nov 26 2019 at 20:34)</a>:</h4>
<p><a href="https://github.com/Mark-Simulacrum/cargo/commit/c8373fbde42777e389c889e8b16a1d5f823e7a68" target="_blank" title="https://github.com/Mark-Simulacrum/cargo/commit/c8373fbde42777e389c889e8b16a1d5f823e7a68">https://github.com/Mark-Simulacrum/cargo/commit/c8373fbde42777e389c889e8b16a1d5f823e7a68</a> should be enough to rebuild cargo so hopefully not too hard</p>



<a name="181975812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181975812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181975812">(Nov 26 2019 at 23:45)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> awesome, thanks!</p>



<a name="181975857"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181975857" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181975857">(Nov 26 2019 at 23:46)</a>:</h4>
<p>my computer with 28 cores got shut down today and is being shipped to me over the thanksgiving break</p>



<a name="181975864"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181975864" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181975864">(Nov 26 2019 at 23:46)</a>:</h4>
<p>so I'llg et back to you next monday on the numbers for that</p>



<a name="181975871"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/181975871" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#181975871">(Nov 26 2019 at 23:46)</a>:</h4>
<p>okay, sounds good</p>



<a name="182367041"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182367041" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182367041">(Dec 02 2019 at 17:15)</a>:</h4>
<p>to make sure I understand this right, <span class="user-mention" data-user-id="116122">@simulacrum</span> <a href="https://github.com/rust-lang/rust/commit/cf92e2ebf129f4fd8075650b21c612d196b7a2f8" target="_blank" title="https://github.com/rust-lang/rust/commit/cf92e2ebf129f4fd8075650b21c612d196b7a2f8">cf92e2ebf129f4fd8075650b21c612d196b7a2f8</a> is a build which is new rayon plus semaphore-for-jobserver, right? no other changes/</p>



<a name="182367246"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182367246" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182367246">(Dec 02 2019 at 17:17)</a>:</h4>
<p>this looks to be a massive improvement</p>



<a name="182368142"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368142" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368142">(Dec 02 2019 at 17:26)</a>:</h4>
<p>Correct, yes</p>



<a name="182368235"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368235" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368235">(Dec 02 2019 at 17:26)</a>:</h4>
<p>Possibly modulo cargo update to latest master but I can't imagine that makes a difference</p>



<a name="182368328"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368328" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368328">(Dec 02 2019 at 17:27)</a>:</h4>
<p>ok so the numbers aren't entirely exonerating</p>



<a name="182368343"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368343" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368343">(Dec 02 2019 at 17:27)</a>:</h4>
<p>but I posted them to <a href="https://hackmd.io/bmB35-7oRzCeGOCanI0SkA?both" target="_blank" title="https://hackmd.io/bmB35-7oRzCeGOCanI0SkA?both">https://hackmd.io/bmB35-7oRzCeGOCanI0SkA?both</a></p>



<a name="182368356"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368356" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368356">(Dec 02 2019 at 17:27)</a>:</h4>
<p>they look massively better than the prior version</p>



<a name="182368357"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368357" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368357">(Dec 02 2019 at 17:27)</a>:</h4>
<p>I found a flag on <code>epoll</code> that might allow us to avoid thundering herd problem with the pipe, too, which might be enough that we can use that for now</p>



<a name="182368387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368387">(Dec 02 2019 at 17:27)</a>:</h4>
<p>oh nice</p>



<a name="182368440"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368440" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368440">(Dec 02 2019 at 17:28)</a>:</h4>
<p>oh -- are you using the right cargo there?</p>



<a name="182368451"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368451" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368451">(Dec 02 2019 at 17:28)</a>:</h4>
<p>you need to use the one from that toolchain</p>



<a name="182368462"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368462" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368462">(Dec 02 2019 at 17:28)</a>:</h4>
<p>I ran <code>cargo +...</code></p>



<a name="182368487"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368487" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368487">(Dec 02 2019 at 17:28)</a>:</h4>
<p>basically what the script snippets say</p>



<a name="182368493"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368493" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368493">(Dec 02 2019 at 17:28)</a>:</h4>
<p>I think rustup-toolchain-install-master doesn't do that though</p>



<a name="182368513"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368513" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368513">(Dec 02 2019 at 17:28)</a>:</h4>
<p>oh fascinating</p>



<a name="182368520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368520">(Dec 02 2019 at 17:28)</a>:</h4>
<p>I may need to fix my thing then</p>



<a name="182368527"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368527" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368527">(Dec 02 2019 at 17:28)</a>:</h4>
<p>i.e., you're not downloading cargo by default</p>



<a name="182368530"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368530" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368530">(Dec 02 2019 at 17:28)</a>:</h4>
<p>also how did this</p>



<a name="182368545"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368545" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368545">(Dec 02 2019 at 17:29)</a>:</h4>
<p>I must be using the equivalent of no jobserver then</p>



<a name="182368556"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368556" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368556">(Dec 02 2019 at 17:29)</a>:</h4>
<p>ok I'll need to recollect data</p>



<a name="182368620"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368620" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368620">(Dec 02 2019 at 17:29)</a>:</h4>
<p>yeah normal cargo would basically be no jobserver (well, cargo itself would limit to 28, but not internal parallelism)</p>



<a name="182368642"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368642" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368642">(Dec 02 2019 at 17:29)</a>:</h4>
<p>I cna't collect numbers right this second, will be a bit</p>



<a name="182368913"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182368913" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182368913">(Dec 02 2019 at 17:32)</a>:</h4>
<p>sounds good -- I think you shouldn't need to build the cargo, but it's probably best to do so -- just to be safe -- the submodule in that commit was updated as needed to an appropriate commit I think</p>



<a name="182369523"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369523" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369523">(Dec 02 2019 at 17:38)</a>:</h4>
<p>ok I think it's working now</p>



<a name="182369530"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369530" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369530">(Dec 02 2019 at 17:38)</a>:</h4>
<p>I downloaded the precompiled cargo</p>



<a name="182369540"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369540" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369540">(Dec 02 2019 at 17:38)</a>:</h4>
<p>and i confirmed that <code>-j3</code> actually limits things</p>



<a name="182369566"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369566" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369566">(Dec 02 2019 at 17:38)</a>:</h4>
<p>(it'll loosely not work due to new-rayon bugs that we spoke about last week)</p>



<a name="182369600"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369600" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369600">(Dec 02 2019 at 17:39)</a>:</h4>
<p>i.e., I think we can get in a situation where we have more than N running rustcs and such, but we will (hopefully) never actually be doing more than N threads of work</p>



<a name="182369906"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369906" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369906">(Dec 02 2019 at 17:42)</a>:</h4>
<p>ok those numbers look even better -- <a href="https://hackmd.io/bmB35-7oRzCeGOCanI0SkA?both" target="_blank" title="https://hackmd.io/bmB35-7oRzCeGOCanI0SkA?both">https://hackmd.io/bmB35-7oRzCeGOCanI0SkA?both</a></p>



<a name="182369933"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369933" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369933">(Dec 02 2019 at 17:42)</a>:</h4>
<p>system time barely changes</p>



<a name="182369968"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182369968" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182369968">(Dec 02 2019 at 17:43)</a>:</h4>
<p><code>top</code> still shows a smidge of red, but nowhere near where it was before</p>



<a name="182370091"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182370091" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182370091">(Dec 02 2019 at 17:44)</a>:</h4>
<p>14 threads seems to be a sweet spot for me</p>



<a name="182370097"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182370097" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182370097">(Dec 02 2019 at 17:44)</a>:</h4>
<p>so not ahuge amount of benefit for hyperthreads</p>



<a name="182370117"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182370117" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182370117">(Dec 02 2019 at 17:44)</a>:</h4>
<p>do you know if you're using LLD by default locally?</p>



<a name="182370198"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182370198" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182370198">(Dec 02 2019 at 17:45)</a>:</h4>
<p>I am not</p>



<a name="182370202"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182370202" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182370202">(Dec 02 2019 at 17:45)</a>:</h4>
<p>gtg now</p>



<a name="182370273"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182370273" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182370273">(Dec 02 2019 at 17:46)</a>:</h4>
<p>okay, was going to suggest disabling threading if yes</p>



<a name="182374573"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182374573" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182374573">(Dec 02 2019 at 18:30)</a>:</h4>
<p>When compiling one crate on 8 cores, SMT made things slower until I landed some PRs which reduced contention. I'm using a Ryzen R7 1700, so I don't know how SMT affect performance on Intel chips, but I know AMD generally performs better in multi-threaded workloads.</p>



<a name="182375746"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182375746" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Santiago Pastorino <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182375746">(Dec 02 2019 at 18:41)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> <a href="https://github.com/rust-lang/rust/compare/master...spastorino:acquire_thread_then_spawn?expand=1" target="_blank" title="https://github.com/rust-lang/rust/compare/master...spastorino:acquire_thread_then_spawn?expand=1">https://github.com/rust-lang/rust/compare/master...spastorino:acquire_thread_then_spawn?expand=1</a></p>



<a name="182375801"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182375801" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Santiago Pastorino <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182375801">(Dec 02 2019 at 18:42)</a>:</h4>
<p>handler code is basically the default handler code and added the acquire and release stuff before and after spawning</p>



<a name="182377861"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182377861" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182377861">(Dec 02 2019 at 19:03)</a>:</h4>
<p>That would release the token before the spawned thread finished, right?</p>



<a name="182379894"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182379894" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Santiago Pastorino <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182379894">(Dec 02 2019 at 19:24)</a>:</h4>
<p>ouch, we want that right after the run call</p>



<a name="182380969"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182380969" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Santiago Pastorino <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182380969">(Dec 02 2019 at 19:36)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> <a href="https://github.com/rust-lang/rust/pull/66972" target="_blank" title="https://github.com/rust-lang/rust/pull/66972">https://github.com/rust-lang/rust/pull/66972</a></p>



<a name="182382188"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182382188" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182382188">(Dec 02 2019 at 19:49)</a>:</h4>
<p><span class="user-mention" data-user-id="116266">@Santiago Pastorino</span> Wouldn't that cause rustc to wait until it has all the tokens before doing anything? Or did you do some Rayon changes to thread spawning too?</p>



<a name="182383173"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182383173" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Santiago Pastorino <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182383173">(Dec 02 2019 at 19:59)</a>:</h4>
<p>did this as request of <span class="user-mention" data-user-id="116122">@simulacrum</span> but we didn't touch rayon thread spawning</p>



<a name="182383416"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182383416" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182383416">(Dec 02 2019 at 20:01)</a>:</h4>
<p>Rayon also acquires tokens when spawning / closing threads, so you'll need to remove those too</p>



<a name="182594527"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182594527" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182594527">(Dec 04 2019 at 20:18)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> I've been looking into the EPOLLEXCLUSIVE improvement I mentioned, and it looks like it's definitely linux-specific (along with all of epoll). It also looks like kqueue on macOS is claimed to avoid the thundering herd problem, but I can find no evidence of this in manpages (just in random blog posts).</p>
<p>Right now as far as I can tell we can just break compatibility with make, and use semaphores (which are supported on macOS and Linux, at least, and are in POSIX so I expect pretty wide support). That's probably not really tenable though. The alternative then is to work on the Cargo integration to have a single process read/write the file descriptor I guess.</p>
<p>Do you feel that working on the Cargo integration would be reasonable? Should we consider 'just' breaking?</p>



<a name="182595026"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595026" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595026">(Dec 04 2019 at 20:23)</a>:</h4>
<p>If we pursue the route of semaphores, here's what I think we should do:</p>



<a name="182595038"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595038" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595038">(Dec 04 2019 at 20:23)</a>:</h4>
<p>well, let me just type it out</p>



<a name="182595131"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595131" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595131">(Dec 04 2019 at 20:24)</a>:</h4>
<p>I think we should add dual support in the <code>jobserver</code> crate for semaphores and pipes. Cargo would then tell the compiler either (a) here's a semaphore, go wild, or (b) here's a pipe jobserver, you can spawn at most N threads where N is like 10.</p>



<a name="182595144"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595144" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595144">(Dec 04 2019 at 20:24)</a>:</h4>
<p>Most compilations with Cargo do not integrate with an external jobserver</p>



<a name="182595153"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595153" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595153">(Dec 04 2019 at 20:24)</a>:</h4>
<p>so that means most compilations would use the semaphore and can go wild</p>



<a name="182595167"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595167" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595167">(Dec 04 2019 at 20:24)</a>:</h4>
<p>for those that do we'd limit rustc's parallelism to alleviate the thundering herd problem</p>



<a name="182595191"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595191" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595191">(Dec 04 2019 at 20:25)</a>:</h4>
<p>and we can have a work item for later to fix this</p>



<a name="182595211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595211">(Dec 04 2019 at 20:25)</a>:</h4>
<p>basically Cargo just needs the ability to query a <code>jobserver::Client</code> if it's a semaphore or a pipe</p>



<a name="182595222"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595222" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595222">(Dec 04 2019 at 20:25)</a>:</h4>
<p>and the <code>jobserver</code> crate auto-detects semaphores or pipes</p>



<a name="182595229"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595229" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595229">(Dec 04 2019 at 20:25)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> does that sound reasonable?</p>



<a name="182595253"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595253" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595253">(Dec 04 2019 at 20:25)</a>:</h4>
<p>I think that should give us a lot of bang for not a lot of buck, while leaving it possible to fix this in the future</p>



<a name="182595325"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595325" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595325">(Dec 04 2019 at 20:26)</a>:</h4>
<p>my impression is that most compilations do shell out to cmake/make somewhere or so which my impression was limits what we can do</p>



<a name="182595358"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595358" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595358">(Dec 04 2019 at 20:27)</a>:</h4>
<p>but yes, that sounds reasonable</p>



<a name="182595374"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595374" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595374">(Dec 04 2019 at 20:27)</a>:</h4>
<p>I can work on a PR to jobserver-rs to make that logic happen</p>



<a name="182595402"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595402" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595402">(Dec 04 2019 at 20:27)</a>:</h4>
<p>(it's worth noting that e.g. on Windows it seems initial measurements suggest that a semaphore there still suffers from the thundering herd problem though we don't have super knowledgeable people testing that yet)</p>



<a name="182595520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595520">(Dec 04 2019 at 20:29)</a>:</h4>
<p>but this does sounds reasonable</p>



<a name="182595539"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595539" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595539">(Dec 04 2019 at 20:29)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> will go ahead and start work on this</p>



<a name="182595634"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595634" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595634">(Dec 04 2019 at 20:30)</a>:</h4>
<p>Oh wait right</p>



<a name="182595644"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595644" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595644">(Dec 04 2019 at 20:30)</a>:</h4>
<p>I forgot that</p>



<a name="182595689"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595689" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595689">(Dec 04 2019 at 20:30)</a>:</h4>
<p>The cmake make stuff</p>



<a name="182595848"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595848" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595848">(Dec 04 2019 at 20:32)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> let's leave this on the backburner for now</p>



<a name="182595853"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595853" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595853">(Dec 04 2019 at 20:32)</a>:</h4>
<p>and fix windows first</p>



<a name="182595856"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595856" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595856">(Dec 04 2019 at 20:32)</a>:</h4>
<p>and see what that requires</p>



<a name="182595879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595879">(Dec 04 2019 at 20:32)</a>:</h4>
<p>if semaphores don't work there we'll likely <em>require</em> a system where cargo manages everything anyway</p>



<a name="182595918"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595918" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595918">(Dec 04 2019 at 20:33)</a>:</h4>
<p>hm, okay</p>



<a name="182595937"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595937" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595937">(Dec 04 2019 at 20:33)</a>:</h4>
<p>so fixing windows is basically a "shrug" from me -- I can look around at documentation for the primitives we use today</p>



<a name="182595962"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182595962" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182595962">(Dec 04 2019 at 20:34)</a>:</h4>
<p>but if those -- already being semaphores -- are still leading to a bunch of wakeups, then there's plausibly nothing we can do</p>



<a name="182596548"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596548" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596548">(Dec 04 2019 at 20:40)</a>:</h4>
<p>well the thing we can do to fix that is what we'd do to fix the integration with the jobserver</p>



<a name="182596557"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596557" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596557">(Dec 04 2019 at 20:40)</a>:</h4>
<p>which is that each rustc instances has a separate ipc mechanism connecting it to cargo</p>



<a name="182596571"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596571" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596571">(Dec 04 2019 at 20:40)</a>:</h4>
<p>and cargo selects which rustc to wake up, only waking up one instead of all of them</p>



<a name="182596592"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596592" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596592">(Dec 04 2019 at 20:40)</a>:</h4>
<p>like there would be N jobservers instead of just 1</p>



<a name="182596594"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596594" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596594">(Dec 04 2019 at 20:40)</a>:</h4>
<p>(ish)</p>



<a name="182596638"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596638" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596638">(Dec 04 2019 at 20:41)</a>:</h4>
<p>but not literally jobservers because cargo has to react to a request for a token</p>



<a name="182596647"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596647" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596647">(Dec 04 2019 at 20:41)</a>:</h4>
<p>oh</p>



<a name="182596660"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182596660" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182596660">(Dec 04 2019 at 20:41)</a>:</h4>
<p>I thought you meant fixing windows without doing the cargo thing</p>



<a name="182598087"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598087" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598087">(Dec 04 2019 at 20:55)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> so do you expect to invest time into trying to fix windows semaphores, and then if that fails, we invest into making cargo be the one to dispatch all jobserver operations?</p>



<a name="182598297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598297">(Dec 04 2019 at 20:56)</a>:</h4>
<p>oh sorry so to clarify, I mean that we should "fix windows" in that we should find <em>some</em> solution that is on the same par of performance as linux</p>



<a name="182598319"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598319" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598319">(Dec 04 2019 at 20:57)</a>:</h4>
<p>I don't think it matters what that is, and it could be the semaphores we have today with a tweak, but we should fix it somehow</p>



<a name="182598334"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598334" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598334">(Dec 04 2019 at 20:57)</a>:</h4>
<p>and then we can go from there to see how to unify all the implementations and land this all for real</p>



<a name="182598341"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598341" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598341">(Dec 04 2019 at 20:57)</a>:</h4>
<p>ah, okay</p>



<a name="182598345"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598345" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598345">(Dec 04 2019 at 20:57)</a>:</h4>
<p>I don't think I'll personally have time to work on this, but I can allocate time if necessary</p>



<a name="182598365"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598365" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598365">(Dec 04 2019 at 20:57)</a>:</h4>
<p>Windows won't really get on par with Linux though. It's pretty known to be slow, especially with multithreading =P</p>



<a name="182598380"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598380" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598380">(Dec 04 2019 at 20:58)</a>:</h4>
<p>makes sense; I can look at the flags to the semaphore primitives we use today at least</p>



<a name="182598435"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182598435" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182598435">(Dec 04 2019 at 20:58)</a>:</h4>
<p>and then failing that take a look at making the cargo refactor</p>



<a name="182600932"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182600932" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> andjo403 <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182600932">(Dec 04 2019 at 21:21)</a>:</h4>
<p>but do not know how mush the jobserver is affecting the times because when I look at eg. the cargo crate then there is no other crate running and I still get longer times with 32 threads compared with 4 threads at least on windows</p>



<a name="182603784"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182603784" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182603784">(Dec 04 2019 at 21:50)</a>:</h4>
<p>There's likely contention with thread spawning in Windows and inside rustc too (miri, symbols, spans, unknown stuff)</p>



<a name="182604428"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182604428" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182604428">(Dec 04 2019 at 21:58)</a>:</h4>
<p><span class="user-mention" data-user-id="116466">@Zoxc</span> fwiw, all the contention during normal execution we were able to detect with some loose benchmarks wasn't really at the syscall level -- of course, this may just be indicative of either bad measurement or lack of good parallelism in the compiler</p>



<a name="182789057"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182789057" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182789057">(Dec 06 2019 at 18:28)</a>:</h4>
<p>as a quick update it looks like there's no better primitive to be using on windows at least that I can see</p>



<a name="182789069"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182789069" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182789069">(Dec 06 2019 at 18:28)</a>:</h4>
<p>(based on MS docs)</p>



<a name="182789147"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/182789147" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#182789147">(Dec 06 2019 at 18:29)</a>:</h4>
<p>I guess then the next step is to start working on making cargo act as the jobserver for rustc instances.. I'll start working on that soon then I suppose, though it feels quite painful</p>



<a name="183595993"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/183595993" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#183595993">(Dec 16 2019 at 22:03)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> One idea I would be to have separate process and thread tokens. We could use semaphores on Linux for the thread tokens. Also if a process gives up all its thread tokens temporarily, it couldn't be used to spawn further processes.</p>



<a name="183596134"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver%20improvements/near/183596134" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/jobserver.20improvements.html#183596134">(Dec 16 2019 at 22:05)</a>:</h4>
<p>we discussed this a bit and came up with a solution which we expect to work -- the gist is that each rustc gets its own jobserver pipe/semaphore/whatever which it'll only read from and just before blocking on read it'll print out a message to cargo asking for a token; cargo will then be the only one acquiring tokens from the real jobserver pipe</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>